Break up one long video into segments without losing quality like this:

ffmpeg -ss 00:01:22 -t 00:04:20 -i VID_120506_394.MP4 -acodec copy -vcodec copy    HBC_Baptisms_Spring2012_1_Intro.MP4

Merge two videos into one (used when I discovered my new camera automatically starts a new file at 16 minutes or about 2 GB):

mencoder -oac pcm -ovc copy HBC_Baptisms_Spring2012_6a_Noah.MP4 HBC_Baptisms_Spring2012_6b_Noah.MP4 -o HBC_Baptisms_Spring2012_6_Noah.MP4

ffmpeg -i HBC_Baptisms_Spring2012_10_ClosingComments.MP4 -an -s hd480 -vcodec libx264 -vpre slow_firstpass -pass 1 -threads 0 -f mp4 -y /dev/null  && ffmpeg -i HBC_Baptisms_Spring2012_10_ClosingComments.MP4 -s hd480 -vcodec libx264 -vpre slow -acodec copy -pass 2 -threads 0 -y HBC_Baptisms_Spring2012_10_ClosingComments.hd480.MP4

ffmpeg -ss 00:05:40 -t 00:02:17 -i VID_120506_394.MP4 -threads 0 HBC_Baptisms_Spring2012_2_Jessica.webm

Before you dismiss these ramblings as nitpicking, let me pose a few questions:

• If you pay for a gallon of gas, would you be content if the pump delivered 3.6 quarts instead of 4.0?
• If someone promises to meet you in an hour, are they "on-time" if they arrive in an hour and 15 minutes?
• If you buy a dozen eggs, do you expect there to be twelve of them?

In all of these areas, we find it reasonable to expect precision in our understanding of each other (even if, as in the example using time, we allow for imprecision in actual performance). This seems to me a quite reasonable expectation.

And yet, for decades, we've accepted in common usage an ambiguity in our descriptions of the memory and disk capacities of computers. And while the discrepancy started small, as disk and memory capacities have grown the discrepancy is growing ever larger on a percentage basis of the sizes involved. And yet, there is an alternative terminology that we could be using for greater precision.

It started innocently enough. Computers are binary devices and computer resources are addressed in binary units. Early computer documentation explicitly specified the (very small) memory and disk size with numbers like 1024, 2048, 4096, 8192 and so on. But as the available sizes grew, our impulse for brevity in speech and writing led to a desire to round these off to larger units. We already were familiar with the metric system prefixes -- kilo for thousand (10^3), mega for million (10^6), and giga for billion (10^9) -- and it was noted that those units were "pretty close" to some units derived from binary units like 1024 (2^10). So it became common to use kilo (sometimes) for 1024 units of computer related things, while it was also used for 1000 units (the strict definition of the prefix). So ambiguity was accepted for convenience -- after all, there's less than 2.5% difference between 1000, and 1024, right?

Some parts of our industry used this discrepancy to their advantage, and to the confusion of their customers. For example, disk drive manufacturers prefer to report the size of their drives in decimal units (because it makes their drives sound larger) despite the fact that virtually all software will report disk sizes in binary units (making the reported size of the drive seem smaller than advertised on the box).

The extent of the ambiguity is considerable. Hard disk manufacturers use the prefixes to reflect decimal values. Operating systems report hard disk sizes using binary values. Flash memory uses decimal values. RAM uses binary values. CD-ROMs are measured in binary values while DVDs are measured in decimal values. And diskette drives (if you can still find one) are measured in a strange hybrid where 1 "megabyte" means 1024 x 1000 bytes. Clock rates, data transfer rates, and network communication speeds are all generally measured in decimal values.

But the larger the capacities we have, the larger (as a percentage) the discrepancy grows. Consider:

A kilobyte:

• Defined as 10^3 is 1,000 bytes
• Defined as 2^10 is 1,024 bytes
• This is a 2.40% discrepancy

A megabyte

• Defined as 10^6 is 1,000,000 bytes
• Defined as 2^20 is 1,048,576 bytes
• This is a 4.86% discrepancy

A gigabyte

• Defined as 10^9 is 1,000,000,000 bytes
• Defined as 2^30 is 1,073,741,824 bytes
• This is a 7.37% discrepancy

A terabyte

• Defined as 10^12 is 1,000,000,000,000 bytes
• Defined as 2^40 is 1,099,511,627,776 bytes
• This is a 9.95% discrepancy

A petabyte

• Defined as 10^15 is 1,000,000,000,000,000 bytes
• Defined as 2^50 is 1,125,899,906,842,620 bytes
• This is a 12.59% discrepancy

An exabyte

• Defined as 10^18 is 1,000,000,000,000,000,000 bytes
• Defined as 2^60 is 1,152,921,504,606,850,000 bytes
• This is a 15.29% discrepancy

A 2.5% difference between what you mean by a term and what I think you mean by a term may be fairly trivial. But a 10%, 12% or 15% difference is not so readily dismissed.

And there is an alternative.

As long ago as 1968, some computer scientists began proposing a separate set of prefixes that refer exclusively to the binary definitions. And as long ago as 1998 a number of standards bodies and trade organizations approved a proposal for and recommended adoption of a new naming convention for these units. The new names for the binary units are based on the decimal names, but replace the second syllable of the corresponding decimal name with "bi" for "binary". So 1000 bytes is a kilobyte but 1024 bytes is a "kibibyte". Under the new (now 14 year-old) proposals, each of the following units would have a unique and unambiguous meaning:

By now (2012) many organizations recommended the use of these conventions (or at least insist that the traditional prefixes refer only to the decimal values), including:

• The International Electrotechnical Commission (IEC)
• The U.S. National Institute of Standards and Technology (NIST)
• The IEEE
• The International Bureau of Weights and Measures (BIPM)
• The Society of Automotive Engineers (SAE)
• The European Committee for Electrotechnical Standardization (CENELEC)

I don't know about you, but I'm sold on this.  Going forward, I'll be more precise about the way I describe memory and disk space.  I hope you'll join me.

A few links for further reading:

http://physics.nist.gov/cuu/Units/binary.html

http://en.wikipedia.org/wiki/Binary_prefix

http://en.wikipedia.org/wiki/International_System_of_Units

http://lpar.ath0.com/2008/07/15/si-unit-prefixes-a-plea-for-sanity/

http://members.optus.net/alexey/prefBin.xhtml

My family and I recently started attending a church (Hutto Bible Church) in the community to which we recently moved.  We're in exciting times there -- today was the church's fourth anniversary, and our third or fourth week in a newly renovated building we bought (we were meeting in a school before).  When the property was purchased, it was thought that we'd have *plenty* of room for growth, but the turnout on our first day in the new building led to the real possibility that, if Easter attendance was as much higher that the attendance on that day (helped along by lots of direct mail pieces and door-hangers in the neighborhood) we could well be standing room only.

Additionally, the local Fire Marshall got word of our high expectations and came 'round to remind the staff that our enthusiasm did not mean that overcrowding could be permitted for this special day.  If we allowed more people in the auditorium than the building was rated for, then we could be fined or have the services shut down.

So... with about two weeks of notice, no budget, and no specialized equipment  we set out to cobble together a streaming video solution so that both our worship and the message could be viewed and heard from an outdoor "overflow venue" (the building we're in was formerly a restaurant and had a covered patio space ideal for that.)

So we put together the following equipment:

• A borrowed camera with a firewire (ieee1394) output
• Two laptops running Ubuntu (one needed a firewire port) with these apps installed:
  • dvgrab -- to pull the raw DV data from the camera over firewire
  • ffmpeg -- to "transcode" the DV video into compressed formats.
  • vlc -- a media player cable of being both a streaming media server and client
• Another laptop running Windows (see... I can do things involving Windows... I just don't have to like it!) and the Windows version of VLC.
• A borrowed HDTV with a VGA input

Dealing with digital video formats can be a real pain -- incompatibilities can arise at several different levels.  You have to choose a streaming method -- each of which is likely to be incompatible with several of the "container formats" or codecs. You have to choose a container format (which may introduce limitations in the choices of streaming methods and codecs). You have to choose audio and video "codecs" that work with the other variables, and you have to put it all together in the rather inscrutable syntax of some very complex applications.  I learned probably a hundred ways it WON'T work.  But here's what finally DID:

• On the computer attached via firewire, we used the following commands:

$ sudo chown .video /dev/raw1394

This allowed the non-privileged user (member of the video group) permission to work with the firewire input.

$ dvgrab -noavc -f dv1 - |vlc - --demux rawdv --sout '#transcode{vcodec=h264,vb=512,scale=1,acodec=mp3,ab=192,channels=2,fps=25.0}:standard{access=http,mux=asf,dst=192.168.1.121:8090}'

This command had two parts.  Dvgrab took the DV data from the camera and passed it, through a standard i/o pipe to vlc.  Then vlc takes the data, changes the encoding, and streams it over the wireless network via http.

• On the 2nd Linux computer, kept near the camera operator as a monitor of the process, we used two different commands in separate gnome-terminal sessions.

$ ffmpeg -i http://192.168.1.121:8090 -vcodec libx264 -acodec libmp3lame EasterService.mp4

This command picked up the stream from the wireless network and recorded it to disk

$ vlc -vvv http://192.168.1.121:8090

This command opened up a viewer to the stream so I could ensure that it was working properly and monitor the terminal window for errors or problems

• In the outdoor venue, we had a windows laptop running vlc and driving the HDTV as a second monitor.  On it, we used the vlc graphical interface to connect to the same stream and display it full screen on the TV.
• From the church's soundboard, we ran an output directly to the camera's mic input.  During worship, we recorded from the camera's mic.  During the message we switched it to take its input from the sound board.

We ended up having some people in the overflow venue for each of two services even though there were still a few available seats indoors.  As it turns out, I don't think the Fire Marshal would have had any concerns even if everyone had been inside.  But it sure did ease our minds to know that we wouldn't have had to turn anyone away.

So... to recap:  Linux and other open source software allowed us to use borrowed equipment to provide high-quality video streaming to avert a situation which could have marred our Easter service.  Net cost to the church:  $0.  

Thanks to all who helped!

I recently passed my Novell CLP (Certified Linux Professional) exam in hopes of achieving the CLP certification – only to discover that it also requires completion of the Novell CLA (Certified Linux Administrator) exam and certification or acceptable equivalent.

Upon investigation, I found that the LPIC1 (Linux Professional Institute Certification  Level 1) is one of the acceptable equivalents.  I had previously passed one of the two required exams for the LPIC1, so I took and passed the second one today.

It may take a few days for the various orgs to get their records updated, but soon we should see the following cascading results:

• LPI will, based on my recent LPI 102 exam added to my older LPI 101 exam, grant me the LPIC1 certification.
• Novell will accept my LPIC1 in place of the CLA, and based on the completion of the CLP exam, grant me the CLP certification.
• Based on the combination of my existing Comptia CTT+ certification and the soon to be granted CLP certification, Novell should also grant me their CNI (Certified Novell Instructor).

So my list of current certifications should soon include: CompTIA CTT+, LPIC1, CompTIA Linux+, Novell CLP, Novell CNI, Red Hat RHCSA, and Red Hat RHCE

Five of those will have been achieved or renewed within this 60 day period.  What fun!

In an earlier post (http://linux_scott.posterous.com/editing-video-from-flip-camcorder-in-linux) I documented some ffmpeg commands for video conversion and manipulation.  Here are a few for the creation and manipulation of video screencasts.

Capturing a high-quality lossless "original" recording:

$ ffmpeg -f alsa -ac 2 -i pulse -f x11grab -r 30 -s 657x435 -i :0.0+1923,762 -acodec pcm_s16le -vcodec libx264 -vpre lossless_ultrafast -threads 0 testrecording.mkv

Converting that lossless original to an MP4:

$ ffmpeg -i testrecording.mkv  -acodec libmp3lame -ab 128k -ac 2 -vcodec libx264  -crf 22 -threads 0 test_final.mp4

Or to a Flash video .flv:

$ ffmpeg -i testrecording.mkv  -acodec libmp3lame -vcodec flv   test_final.flv

Or to a Flash video .swf:

$ ffmpeg -i testrecording.mkv  -acodec libmp3lame -vcodec flv   test_final.swf

Someone asked me what tools I use in  developing multimedia content.  Here are most of them…

I'm using the tools listed below.  All of them are free of charge and open source.  Those identified as cross-platform are also available for use from Windows systems.  Even those that are not available within windows natively, could be made available to windows users through Live-DVDs or through virtual machines for those unfortunates without access to a Linux system.

The final two items on the list are not client applications, but .flv players for use on a webserver.

In place of Photoshop, 

Gimp, http://www.gimp.org (Opensource, cross-platform)

Imagemagick, http://www.imagemagick.org/script/index.php (Opensource, cross-platform)

In place of Captivate, 

vnc2swf http://www.unixuser.org/~euske/vnc2swf/ (Opensource, cross-platform)

Salasaga, http://www.salasaga.org/ (Opensource, cross-platform ?)

GTK-recordMyDesktop http://recordmydesktop.sf.net/ (Opensource)

Istanbul http://live.gnome.org/Istanbul (Opensource)

byzanz-record http://www.gnomefiles.org/app.php/Byzanz (Opensource)

I work with Linux in a training environment where I manage multiple systems remotely through SSH and those systems are reinstalled more often than is typical in production use.  Not infrequently, my SSH connection attempts are met with:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
ec:2d:1c:1a:ed:7c:71:ff:4b:c8:7b:46:72:a3:74:b5.
Please contact your system administrator.
Add correct host key in /home/scott/.ssh/known_hosts to get rid of this message.
Offending key in /home/scott/.ssh/known_hosts:82
RSA host key for [10.37.112.161]:3389 has changed and you have requested strict checking.
Host key verification failed.

The fix is easy, and detailed in the message above -- just delete line #82 of  ~/.ssh/known_hosts.  But that means invoking an editor, navigating to line #82, making sure it is the right line and then deleting it.  How about something quick and easy from the command line instead?  Something like:

sed -i 82d ~/.ssh/known_hosts

That's one I expect to use frequently going forward!

Wanted to share a few tips on using video from the FLIP UltraHD. It's a great camera but, as is usually the case, it is Windows and Mac who get all the love from the manufacturer in terms of application support -- so if you want to edit the videos in any way you need to know your way around a few native Linux video editing tools and applications.

The Video Format

• h.264 video encoding  
• aac audio encoding 
• MPEG4 container 
• 1280x720 resolution 
• BIG Files! 

The Editing Tools

• Kino will convert to raw DV, edit, and export back to other formats.
• ffmpeg will convert in almost any conceivable way!

Some Commands I've Found Useful

Scale a group of videos as a batch:

$ for f in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 ; do ffmpeg -i VID000$ {f}.MP4 -s hd480 -vcodec libx264 -acodec libmp3lame reds${f}_264.mp4; done

Converts a group of videos from 1280x720 to 852x480 -- reducing the file size tremendously (a 330 MB file was reduced to 50 MB) while still retaining a very nice picture.    

Excise, Scale, and Rename a Clip:

$ ffmpeg -ss 00:02:05 -t 00:02:25 -i VID00003.MP4 -s hd480 -vcodec libx264 - acodec libmp3lame BearAwards_hd480.MP4

Excises just a portion (2 minutes and 25 seconds long, beginning at 2:05 into the video) of a longer video while also resizing it and renaming it.

Convert a wide-format (hd480) video into a letterboxed DVD-ready format:

$ ffmpeg -threads 2 -i VID00041_hd480.mp4 -target ntsc-dvd -s 720x404 -padtop 38 -padbottom 38 VID00041_dvd.mpg

Show the formats supported by ffmpeg:

$ ffmpeg -formats

ffmpeg functionality may vary from system to system depending on how you (or your distro's package maintainer) compiled it. This command shows which formats are available to you.

Convert from one format to another:

$ ffmpeg -i VID00006.MP4 -s hd480 -vcodec flv -acodec libmp3lame reds06.flv $ ffmpeg -i VID00006.MP4 -s hd480 -vcodec flv -acodec libmp3lame reds06.swf

Converts from MPEG-4 to Flash formats. I don't pretend to have this optimized for any particular purpose though, so other options may be preferable.:

Convert from uncompressed DV back to MP4:

$ ffmpeg -i reds01_264.mp4.dv -s hd480 -vcodec libx264 -acodec libmp3lame fromDV_264.mp4

Kino only edits in DV format. So, to edit the files created by the FLIP, you must import them -- which creates enormous files in .dv format. After editing, you'll want them back in a compressed format.

Now, Go Learn More

That just scratches the surface, but already I've been able to make my videos dance to my limited tune. Good luck as you learn more!

What a difference a day makes.  Yesterday I had no indication that anyone at AT&T was listening at all (despite their presence on Twitter with @attnews).  But right away this morning I was contacted by Seth Bloom (@sethbloom) who was going to forward my concerns to someone involved in their Customer service department.  We corresponded by email and he took my phone number to pass on.

Moments ago, I received a call from Mattie Cheeks.  She was quite conciliatory, agreed with several of the points I've raised and expressed an intent to work to improve their support in these areas.  I expressed my eagerness to provide an update someday detailing the changes that have been made. 

A few years ago, I would have dismissed such promises for the empty platitudes they would have been -- no company the size of AT&T was going to implement changes in response to one disgruntled customer.  But today, with the increasing awareness by many companies of the power social media has to promote or to wound their brand and with examples of positive outcome from responding to customers, I'm cautiously optimistic.

So, at least for now, I'm staying with AT&T and Uverse.  Their prices are the best available in my area.  But I also intend to be a gadfly until I can consider the support I receive from AT&T to be as solid as what I would receive from their competitors.

And I'll try to keep you posted along the way.

Scott

I called them on my lunch break today -- navigated through the voice response system and was deposited in a queue with only a steady busy signal.

Later, I tried again and spoke with Uverse technical support. No one last night had clearly stated whether I would have to call in to get service re-established, or whether it would happen automatically when the payment was recognized. So I explained the situation and the agent checked the account. It was still listed as disabled.

I asked him to do what he had to to get it taken care of. He explained that he couldn't change that status -- that that could only be done by someone in billing.

Okay... so put me through to them!

He said, no -- instead, I should just wait a few hours and see if it fixes itself...

How about NO!

Finally, I'm connected to billing and they get it taken care of in fairly short order. BUT, it turns out that it would NOT have been automatically taken care of in just a few hours. Had I taken the advice of the agent, I'd be w/o service still.

We have it straightened out for now. I'm just spoiled by Time Warner. With them, I called them for various problems and was NEVER told that I'd have to wait till morning (or worse... till Monday) because some office or another was closed.

Not sure I'm willing to stay with a company willing to allow an outage to persist so long when it is entirely a matter of software settings to resolve it (no need to send a technician or anything)... No one was willing to take my money (TWC will take your money any time -- it's what they're best at!) and even when I paid via the web and provided a confirmation number they could not turn me back on -- just because they don't give anyone who works nights the permission to click the right button!

So what would I like to see from AT&T?  What would make me SATISFIED to remain as their customer?

• at least an escalation path so that SOMEWHERE on the night shift would be someone who could take a payment and re-enable the service -- or at the very least, re-enable the service upon receipt of a confirmation number, or provisionally re-enable the service until the morning crew arrives.

• Better informed and more helpful agents.  Several times I was misled about what would be required.  Several other times they were less than fully forthcoming with information about what I needed to do to resolve this.

Do I hate AT&T?  No, I just want them to be better!  For all of us.  Just as a blogger got the attention of Dell with his column on "Dell Hell" and that led to some real, substantial, and lasting changes in Dell's customer service, I hope that someone at AT&T will take seriously my criticisms and work to make it better.  That aforementioned blogger (Was his name Mark Jarvis?  I don't remember for certain and don't want to look it up right now.) has since become, at least on occasion, an enthusiastic advocate of Dell and the changes they've made in visibly listening to customers.  I'd love to someday be in the equivalent position for AT&T.  Are you listening, AT&T?  Let's talk!

(Disclosure: I work for Dell, but don't speak for it in any way.  These views are mine alone.)